Example: Binary classification¶
This example shows how to use ATOM to solve a binary classification problem. Additonnaly, we'll perform a variety of data cleaning steps to prepare the data for modeling.
The data used is a variation on the Australian weather dataset from Kaggle. You can download it from here. The goal of this dataset is to predict whether or not it will rain tomorrow training a binary classifier on target RainTomorrow.
Load the data¶
In [1]:
Copied!
# Import packages
import pandas as pd
from atom import ATOMClassifier
# Import packages
import pandas as pd
from atom import ATOMClassifier
In [2]:
Copied!
# Load data
X = pd.read_csv("docs_source/examples/datasets/weatherAUS.csv")
# Let's have a look
X.head()
# Load data
X = pd.read_csv("docs_source/examples/datasets/weatherAUS.csv")
# Let's have a look
X.head()
Out[2]:
| Location | MinTemp | MaxTemp | Rainfall | Evaporation | Sunshine | WindGustDir | WindGustSpeed | WindDir9am | WindDir3pm | ... | Humidity9am | Humidity3pm | Pressure9am | Pressure3pm | Cloud9am | Cloud3pm | Temp9am | Temp3pm | RainToday | RainTomorrow | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | MelbourneAirport | 18.0 | 26.9 | 21.4 | 7.0 | 8.9 | SSE | 41.0 | W | SSE | ... | 95.0 | 54.0 | 1019.5 | 1017.0 | 8.0 | 5.0 | 18.5 | 26.0 | Yes | 0 |
| 1 | Adelaide | 17.2 | 23.4 | 0.0 | NaN | NaN | S | 41.0 | S | WSW | ... | 59.0 | 36.0 | 1015.7 | 1015.7 | NaN | NaN | 17.7 | 21.9 | No | 0 |
| 2 | Cairns | 18.6 | 24.6 | 7.4 | 3.0 | 6.1 | SSE | 54.0 | SSE | SE | ... | 78.0 | 57.0 | 1018.7 | 1016.6 | 3.0 | 3.0 | 20.8 | 24.1 | Yes | 0 |
| 3 | Portland | 13.6 | 16.8 | 4.2 | 1.2 | 0.0 | ESE | 39.0 | ESE | ESE | ... | 76.0 | 74.0 | 1021.4 | 1020.5 | 7.0 | 8.0 | 15.6 | 16.0 | Yes | 1 |
| 4 | Walpole | 16.4 | 19.9 | 0.0 | NaN | NaN | SE | 44.0 | SE | SE | ... | 78.0 | 70.0 | 1019.4 | 1018.9 | NaN | NaN | 17.4 | 18.1 | No | 0 |
5 rows × 22 columns
Run the pipeline¶
In [3]:
Copied!
# Call atom using only 5% of the complete dataset (for explanatory purposes)
atom = ATOMClassifier(X, y="RainTomorrow", n_rows=0.05, n_jobs=8, verbose=2)
# Call atom using only 5% of the complete dataset (for explanatory purposes)
atom = ATOMClassifier(X, y="RainTomorrow", n_rows=0.05, n_jobs=8, verbose=2)
<< ================== ATOM ================== >> Configuration ==================== >> Algorithm task: Binary classification. Parallel processing with 8 cores. Parallelization backend: loky Dataset stats ==================== >> Shape: (7109, 22) Train set size: 5688 Test set size: 1421 ------------------------------------- Memory: 1.25 MB Scaled: False Missing values: 15868 (10.1%) Categorical features: 5 (23.8%) Duplicates: 1 (0.0%)
In [4]:
Copied!
# Impute missing values
atom.impute(strat_num="median", strat_cat="drop", max_nan_rows=0.8)
# Impute missing values
atom.impute(strat_num="median", strat_cat="drop", max_nan_rows=0.8)
Fitting Imputer... Imputing missing values... --> Dropping 7 samples for containing more than 16 missing values. --> Imputing 23 missing values with median (11.9) in column MinTemp. --> Imputing 10 missing values with median (22.6) in column MaxTemp. --> Imputing 72 missing values with median (0.0) in column Rainfall. --> Imputing 3059 missing values with median (4.6) in column Evaporation. --> Imputing 3382 missing values with median (8.5) in column Sunshine. --> Dropping 467 samples due to missing values in column WindGustDir. --> Imputing 466 missing values with median (39.0) in column WindGustSpeed. --> Dropping 479 samples due to missing values in column WindDir9am. --> Dropping 165 samples due to missing values in column WindDir3pm. --> Imputing 53 missing values with median (13.0) in column WindSpeed9am. --> Imputing 115 missing values with median (17.0) in column WindSpeed3pm. --> Imputing 72 missing values with median (70.0) in column Humidity9am. --> Imputing 164 missing values with median (52.0) in column Humidity3pm. --> Imputing 699 missing values with median (1017.7) in column Pressure9am. --> Imputing 699 missing values with median (1015.4) in column Pressure3pm. --> Imputing 2698 missing values with median (5.0) in column Cloud9am. --> Imputing 2903 missing values with median (5.0) in column Cloud3pm. --> Imputing 32 missing values with median (16.7) in column Temp9am. --> Imputing 116 missing values with median (21.1) in column Temp3pm. --> Dropping 72 samples due to missing values in column RainToday.
In [5]:
Copied!
# Encode the categorical features
atom.encode(strategy="Target", max_onehot=10, infrequent_to_value=0.04)
# Encode the categorical features
atom.encode(strategy="Target", max_onehot=10, infrequent_to_value=0.04)
Fitting Encoder... Encoding categorical columns... --> Target-encoding feature Location. Contains 47 classes. --> Target-encoding feature WindGustDir. Contains 16 classes. --> Target-encoding feature WindDir9am. Contains 16 classes. --> Target-encoding feature WindDir3pm. Contains 16 classes. --> Ordinal-encoding feature RainToday. Contains 2 classes.
In [6]:
Copied!
# Train an Extra-Trees and a Random Forest model
atom.run(models=["ET", "RF"], metric="f1", n_bootstrap=5)
# Train an Extra-Trees and a Random Forest model
atom.run(models=["ET", "RF"], metric="f1", n_bootstrap=5)
Training ========================= >> Models: ET, RF Metric: f1 Results for ExtraTrees: Fit --------------------------------------------- Train evaluation --> f1: 1.0 Test evaluation --> f1: 0.5956 Time elapsed: 1.414s Bootstrap --------------------------------------- Evaluation --> f1: 0.5709 ± 0.0198 Time elapsed: 1.020s ------------------------------------------------- Time: 2.434s Results for RandomForest: Fit --------------------------------------------- Train evaluation --> f1: 1.0 Test evaluation --> f1: 0.6124 Time elapsed: 0.337s Bootstrap --------------------------------------- Evaluation --> f1: 0.5802 ± 0.0111 Time elapsed: 1.281s ------------------------------------------------- Time: 1.618s Final results ==================== >> Total time: 4.225s ------------------------------------- ExtraTrees --> f1: 0.5709 ± 0.0198 ~ RandomForest --> f1: 0.5802 ± 0.0111 ~ !
Analyze the results¶
In [7]:
Copied!
# Let's have a look at the final results
atom.results
# Let's have a look at the final results
atom.results
Out[7]:
| f1_train | f1_test | time_fit | f1_bootstrap | time_bootstrap | time | |
|---|---|---|---|---|---|---|
| ET | 0.8503 | 0.5688 | 1.414043 | 0.570892 | 1.019728 | 2.433771 |
| RF | 0.8552 | 0.5612 | 0.336765 | 0.580178 | 1.281000 | 1.617765 |
In [8]:
Copied!
# Visualize the bootstrap results
atom.plot_results(title="RF vs ET performance")
# Visualize the bootstrap results
atom.plot_results(title="RF vs ET performance")
In [9]:
Copied!
# Print the results of some common metrics
atom.evaluate()
# Print the results of some common metrics
atom.evaluate()
Out[9]:
| accuracy | ap | ba | f1 | jaccard | mcc | precision | recall | auc | |
|---|---|---|---|---|---|---|---|---|---|
| ET | 0.8478 | 0.6904 | 0.7059 | 0.5688 | 0.3974 | 0.5108 | 0.7750 | 0.4493 | 0.8561 |
| RF | 0.8405 | 0.6775 | 0.7038 | 0.5612 | 0.3901 | 0.4891 | 0.7283 | 0.4565 | 0.8502 |
In [10]:
Copied!
# The winner attribute calls the best model (atom.winner == atom.rf)
print(f"The winner is the {atom.winner.name} model!!")
# The winner attribute calls the best model (atom.winner == atom.rf)
print(f"The winner is the {atom.winner.name} model!!")
The winner is the RF model!!
In [11]:
Copied!
# Visualize the distribution of predicted probabilities
atom.winner.plot_probabilities()
# Visualize the distribution of predicted probabilities
atom.winner.plot_probabilities()
In [12]:
Copied!
# Compare how different metrics perform for different thresholds
atom.winner.plot_threshold(metric=["f1", "accuracy", "ap"], steps=50)
# Compare how different metrics perform for different thresholds
atom.winner.plot_threshold(metric=["f1", "accuracy", "ap"], steps=50)